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Abstract 


I  ED  (improvised  explosive  device ):  programmatically,  an  unintended  consequence  or  im¬ 
pediment  that  can  blow  up  a  development  program. 

Large-scale  systems  (LSS)  being  acquired  by  the  Department  of  Defense  (DoD)  are  frequently 
exemplified  by  the  creation  of  multiple  prime  items,  acquired  under  separate  contract.  The  mul¬ 
tiple  prime  items  are  often  controlled  by  different  organizations,  with  attendant  variations  in  time¬ 
lines  and  funding  stability.  In  most  cases,  each  of  the  prime  items  is  software-intensive.  LSS  are 
encountered  in  several  domains,  including  space-based  systems  and  multi-platform  systems  such 
as  the  Army’s  Future  Combat  System.  These  are  often  referred  to  as  transformational  systems. 

The  concepts  of  time  certain  development  and  incremental  deployment  of  capabilities  would  ap¬ 
pear  to  represent  a  fundamental  change  in  the  programmatic  environment  in  which  LSS  are  ac¬ 
quired.  Such  programs  need  a  “roadmap”  for  acquisition  that  addresses  this  new  enviromnent. 

This  paper  explores  how  continued  use  of  the  existing  acquisition  roadmaps  opens  up  the  poten¬ 
tial  for  running  into  program  pitfalls  (programmatic  IEDs)  that  aren’t  acknowledged  on  the  map  at 
hand. 
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1  Introduction 


The  concepts  of  time  certain  development  and  incremental  development  have  not  yet  been  frilly 
digested  and  interpreted  for  large-scale  systems  (LSS),  but  the  concepts  have  the  effect  of  provid¬ 
ing  a  decision  hierarchy  for  program  direction,  where  time  is  mandated  to  be  a  predetermined  in¬ 
terval  post  Milestone  A,  cost  is  primarily  legislated  based  on  Department  of  Defense  (DoD)  rec¬ 
ommendations  and  the  action  of  external  entities,  and  the  actual  capabilities  to  be  delivered  (i.e., 
requirements)  are  a  matter  of  some  negotiation.  It  is  important  to  recognize  that  “time  certain” 
decision  criteria  and  the  order  in  which  they  are  applied  are  different  from  traditional  models  em¬ 
ployed  on  acquisitions  to  date,  which  is  roughly  to  address  requirements  as  a  whole,  with  negotia¬ 
tion  focused  upon  cost  as  an  independent  variable  (CAIV)  processes.  The  upshot  is  that  what  is 
“optimal”  using  the  two  different  decision  models  isn’t  necessarily  identical. 

At  this  time,  it  is  not  certain  that  the  recommendations  of  Kadish  will  be  embraced  and  mandated 
by  the  DoD  [Kadish  2005].  Regardless,  it  is  important  that  LSS  program  teams  analyze  the  effects 
of  such  a  set  of  potential  mandates.  For  example,  how  big  a  change  is  this  in  practical  terms?  If  an 
LSS  development  program  already  in  process  continues  on  its  current  path  and  the  mandates  are 
imposed,  what  are  the  downside  risks  to  the  LSS?  Will  there  be  a  grandfather  provision  for  pro¬ 
grams  already  under  way?  Suppose  other  major  programs  follow  the  mandates  and  the  LSS  con¬ 
tinues  on  its  current  path:  does  this  make  the  program’s  relations  with  the  DoD  and  Congress 
more  problematic?  These  are  business  questions  that  are  best  addressed  by  senior  leadership. 

The  rest  of  this  paper  addresses  programmatic  IEDs  that  an  LSS  program  is  likely  to  encounter. 
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Programmatic  lEDs 


2.1  What  Criteria  and  Processes  Will  be  Used  to  Determine  What  is  “Operationally 
Useful?” 

A  key  proposition  in  Kadish  is  to  mandate  that  “Operationally  Useful  Capabilities”  be  delivered 
six  years  post  Milestone  A  [Kadish  2005].  Even  if  an  LSS  program  has  made  extraordinary  efforts 
to  engage  the  user  community,  it  is  important  to  revisit  some  fundamental  questions  should  the 
recommendations  in  Kadish  be  accepted  and  enforced  by  the  DoD.  Who  decides  what  is 
“operationally  useful?”  Who  is  the  authoritative  source  for  information?  Has  the  LSS 
identified/cultivated  a  set  of  users  who  can  “think  transformational”  and  envision  the  operational 
value  of  incremental  introduction  of  LSS  capabilities?  In  Kadish,  the  strong  suggestion  is  that 
combatant  commanders  are  the  proper  source.  As  LSS  programs  engage  with  the  users,  the  user’s 
perspective  of  what  is  “useful”  will  change  as  their  understanding  of  what  can  be  done 
operationally  becomes  more  sophisticated.  As  a  result,  the  concept  of  operations  (CONOPS)  and 
doctrine  may  change,  affecting  the  requirements  to  be  deployed  in  a  given  block  or  increment. 
There  is  clearly  an  educational  role  that  LSS  program  personnel  can  fill  to  help  the  user 
community  envision  what  a  force  can  accomplish  with  the  capabilities  that  the  LSS  promises. 

2.2  CONOPS  Varies  Across  Increments 

Consider  a  space-based  LSS,  composed  of  multiple  satellites.  That  which  is  operationally  feasible 
after  first  launch  will  differ  from  that  after  second  launch,  and  so  forth.  If  the  constellation  is 
made  up  of  satellites  of  differing  capability,  this  diversity  will  strongly  influence  the  details  of 
constellation  operations,  and  may  unfortunately  have  a  direct  effect  on  CONOPS.  This  can  thwart 
efforts  to  ensure  that  end  users  can  think  in  terms  of  capability  delivered  to  end  users  without  hav¬ 
ing  to  address  details  of  constellation  operations  that  are  usually  the  domain  of  the  satellite  opera¬ 
tors.  Differences  between  satellites  may  lead  to  design  variations  to  mitigate  the  deltas.  Some 
functions  that  may  be  carried  out  in  hardware  (application-specific  integrated  circuits  and  the  like) 
in  satellites  2,  3,  and  so  forth  may  be  executed  in  software  in  satellite  1.  Do  we  provide  additional 
processor  bandwidth  and/or  memory  margin  to  accommodate  the  expectation  for  more  software 
workarounds  on  the  less  capable  satellite  1  ?  In  communications  satellite  systems,  the  existence  of 
large  inventories  of  legacy  terminals  also  complicates  CONOPS;  it  is  not  unusual  for  full  opera¬ 
tional  capability  envisioned  to  be  deferred  until  the  legacy  inventory  has  been  replaced. 

2.3  Traditional  Systems  Engineering  Practices  May  Not  be  Entirely  Appropriate  to  the 
Incremental  Development  Environment 

Conventional  hierarchical  program  master  schedules  and  plans  may  not  capture  the  dynamics  of 
the  incremental  environment.  In  time  certain  and  incremental  development  the  task  is  to  craft  a 
sequence  of  increments  where  increasing  capability  is  delivered  to  the  field,  some  of  which  may 
actually  be  below  threshold  levels  or  absent  altogether  from  the  first  increment  and  intermediate 
upgrades.  In  our  space-based  example,  the  hardware  and  actual  capabilities  of  distinct  satellites 
may  be  different.  The  staging  of  requirements  packages  that  match  user  expectations — and  can 
grow  from  increment  to  increment — is  a  non-trivial  exercise;  it  requires  that  requirements  and 
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configuration  and  version  interdependencies  be  understood  at  a  deeper  level  than  contained  in  a 
static  requirements  document.  Allocation  splits  between  hardware  and  software  may  change  from 
increment  to  increment. 

The  forces  in  action  also  include  requirements  fluidity,  dynamics  of  the  trade  spaces  being  inves¬ 
tigated  in  parallel  developmental  efforts,  and  evolution  of  external  systems  in  parallel  develop¬ 
ment  (such  as  the  Global  Information  Grid).  The  effect  too  easily  leads  to  a  constant  reconstruc¬ 
tion  of  static  plans  that  are  invalidated  in  short  order  by  changes  in  user  expectations,  changes  in 
the  trade  space,  and  other  factors  which  take  management  attention  away  from  managing  the 
process  of  converging  on  a  set  of  operationally  useful  capabilities  that  can  be  deployed  in  the  ex¬ 
isting  time  and  cost  constraints. 

A  common  technique  to  mitigate  this  problem  area  is  to  institute  a  concurrent  development  ap¬ 
proach.  It  is  customary  in  such  situations  to  provide  a  venue  and  processes  to  address  cross¬ 
domain  systems  engineering  issues,  particularly  the  coordination  of  requirements  and  interfaces. 

2.4  Unrecognized  Software  Development  Dependencies 

Software  development  in  each  of  the  prime  items  of  an  LSS  is  dependent  on  software  develop¬ 
ment  efforts  development  in  the  other  segments.  Inter-segment  interfaces  only  describe  the  nature 
of  the  messages  exchanged  among  the  segments,  but  do  not  ensure  that  the  right  thing  happens  in 
each  segment.  This  problem  is  amplified  in  incremental  development,  since  the  negotiation  to 
arrive  at  what  is  operationally  useful  has  to  span  all  segments.  Configuration  control  is  a  particu¬ 
larly  difficult  issue  that  spans  development  and  sustainment.  The  fact  that  configuration  control 
during  sustainment  and  development  is  executed  by  different  organizations  with  goals  that  are  not 
identical,  with  sustainment  organizations  emphasizing  modifications  to  support  near-term  opera¬ 
tional  needs,  while  development  organizations  emphasize  forward-looking  capabilities.  During 
development,  a  greater  emphasis  on  coordination  across  prime  items  that  comprise  the  LSS  is  ne¬ 
cessary  to  create  an  LSS  that  meets  expectations. 

2.5  Potential  for  Reuse  and  Commercial  Off-the-Shelf  Software  May  Change 

The  new  development  enviromnent  potentially  places  additional  challenges  on  the  use  of  com¬ 
mercial  off-the-shelf  software  (COTS)  and  reusable  code.  Each  increment  must  provide  opera¬ 
tionally  useful  capabilities.  Due  to  the  expected  evolving  nature  of  the  system,  and  the  unknown 
breakdown  of  the  functionality,  what  COTS  can  be  leveraged  and  what  algorithms  and  code  may 
be  reused  will  need  to  constantly  be  re-examined.  This  reveals  a  potential  conflict  between  the 
proposed  time  certain  guidance  and  existing  mandates  to  maximize  employment  of  COTS  and 
reused  software. 

2.6  Inter-Increment  Dependencies 

Each  increment  must  be  considered  an  entire  system  unto  itself,  requiring  the  full  spectrum  of 
software  and  systems  engineering  attention.  While  this  poses  little  difference  from  the  current 
plan  for  Increment  1 ,  it  places  subsequent  increments  in  a  position  to  treat  the  previous  increments 
as  somewhat  of  a  legacy  system  that  has  to  be  accommodated  in  these  subsequent  increment  de¬ 
velopment  efforts.  This  suggests,  for  example,  that  a  full  Increment  1  architectural  suite  will  be 
required,  and  then  something  equivalent  will  be  required  for  each  of  the  following  increments. 
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This  would  infer  a  set  of  parallel  or  staggered  development  efforts,  each  with  its  own  set  of  devel¬ 
opment  artifact  needs  (architectures,  requirements,  transition  plans,  etc.)-  This  staggered  but  near¬ 
ly  in  parallel  set  of  development  projects,  one  for  each  increment,  can  place  unexpected  strains  on 
resources  that  may  need  to  be  shared  (or  not)  between  various  increment  development  efforts.  It  is 
possible  that  other  programs  have  addressed  this  issue,  and  their  experience  may  be  useful  in  for¬ 
mulating  a  specific  strategy  for  a  given  LSS  program. 

2.7  Unprecedented  Software  Integration  Complexity  and  Scale 

In  contrast  to  the  other  items  mentioned,  this  risk  is  independent  of  whether  time  certain  or  incre¬ 
mental  are  chosen  as  a  management  approach,  or  the  status  quo  is  retained.  Most  LSS  programs 
envision  the  integration  of  107or  more  equivalent  lines  of  code  (ELOC)  for  the  total  program.  We 
are  unaware  of  any  DoD  program  in  the  past  20  years  that  has  integrated  software  of  this  size  and 
complexity  within  anywhere  close  to  the  originally  proposed  cost,  schedule,  and  performance  pa¬ 
rameters.  Other  notable  risks  (such  as  sensor  maturity)  can  be  mitigated  in  part  through  trade  stu¬ 
dies,  technology  maturation,  and  negotiations  of  the  operationally  useful  package  of  capabilities. 
The  integration  risk  noted  cannot  be  mitigated  through  any  of  the  mechanisms  currently  employed 
on  the  program.  In  current  plans  the  effect  of  this  risk  will  surface  in  later  stage  integration  and 
test,  a  time  when  the  program  will  again  be  vulnerable  in  a  very  public  way.  It  is  important  that 
any  LLS  program  have  a  vigorous  systems  integration  IPT  (integrated  product  team)  empanelled 
from  the  beginning  of  the  program,  specifically  charged  to  address  the  downstream  integration 
issues,  including  software  integration.  Some  acquisition  organizations  are  attempting  to  adopt  an 
enterprise  approach  to  the  various  programs  in  their  portfolios.  One  tactic  employed  is  to  institute 
cross-program  configuration  control  at  the  requirements  level  to  aid  in  managing  one  of  the  pri¬ 
mary  drivers  of  complexity  growth.  The  management  of  interactions  among  the  prime  items  in 
development  or  in  service  is  also  addressed  by  the  institution  of  test  and  integrations  teams 
charged  with  ensuring  that  dependencies  are  properly  tested  and  existing  operational  capabilities 
are  not  interrupted. 
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